There's a library called p-limit that is built for this purpose, but it's written in ESM, so it's a hassle to deal with. I figured, how hard could it be to implement my own? So I came up with this implementation:
(async() => {
const promisedAxiosPosts = _.range(0, 100).map(async(item, index) => {
console.log(`${index}: starting`);
return Promise.resolve();
});
let i = 0;
for (const promisedAxiosPostGroup of _.chunk(promisedAxiosPosts, 10)) {
console.log(`***********************
GROUP ${i}
SIZE ${promisedAxiosPostGroup.length}
***********************`);
await Promise.all(promisedAxiosPostGroup);
i++;
}
}
)().catch((e) => {
throw e;
})<script src="https://cdnjs.cloudflare.com/ajax/libs/lodash.js/4.17.21/lodash.min.js" integrity="sha512-WFN04846sdKMIP5LKNphMaWzU7YpMyCU245etK3g/2ARYbPK9Ub18eG+ljU96qKRCWh+quCY7yefSmlkQw1ANQ==" crossorigin="anonymous" referrerpolicy="no-referrer"></script>Why isn't this waiting for each chunk to complete before moving on to the next one?
I think that map could be the culprit, but I don't see how: it returns a Promise<void>[]; if it's awaiting on the functions, wouldn't it be returning a void[] (not sure if that's even a thing)?
For this to work, you need to return a function that returns a promise when called. The function (a thunk) delays the execution of the actual action.
After chunking the array, call the functions in the current chunk, and use Promise.all() to wait for all the promises to resolve:
(async() => {
const pendingPosts = _.range(0, 100).map((item, index) => {
return () => { // the thunk
console.log(`${index}: starting`);
// a simulation of the action - an api call for example
return new Promise(resolve => {
setTimeout(() => resolve(), index * 300);
});
}
});
let i = 0;
for (const pendingChunk of _.chunk(pendingPosts, 10)) {
console.log(`***********************
GROUP ${i}
SIZE ${pendingChunk.length}
***********************`);
await Promise.all(pendingChunk.map(p => p())); // invoke the thunk to call the action
i++;
}
}
)().catch((e) => {
throw e;
}).as-console-wrapper { max-height: 100% !important; top: 0; }<script src="https://cdnjs.cloudflare.com/ajax/libs/lodash.js/4.17.21/lodash.min.js" integrity="sha512-WFN04846sdKMIP5LKNphMaWzU7YpMyCU245etK3g/2ARYbPK9Ub18eG+ljU96qKRCWh+quCY7yefSmlkQw1ANQ==" crossorigin="anonymous" referrerpolicy="no-referrer"></script>